Data Preservation in High Energy Physics

Data from high-energy physics (HEP) experiments are collected with significant financial and human effort and are mostly unique. At the same time, HEP has no coherent strategy for data preservation and re-use. An inter-experimental Study Group on HEP data preservation and long-term analysis was convened at the end of 2008 and held two workshops, at DESY Large data sets accumulated during many years of detector operation at particle accelerators are the heritage of experimental high-energy physics (HEP). These data sets offer unique opportunities for future scientific studies, sometimes long after the shutdown of the actual experiments: new theoretical input; new experimental results and analysis techniques; the quest for high-sensitivity combined analyses; the necessity of cross checks. In many cases, HEP data sets are unique; they cannot and most likely will not be superseded by data from newer generations of experiments. Once lost, or in an unusable state, HEP data samples cannot be reasonably recovered. The cost of conserving this heritage through a collaborative, target-oriented long-term data preservation program would be small, compared to the costs of past experimental projects or to the efforts to redo experiments. However, this cost is not negligible, especially for collaborations close or past their end-date. The preservation of HEP data would provide today's collaborations with a secure way to complete their data analysis and enable them to seize new scientific opportunities in the coming years. The HEP community will benefit from preserved data samples through re-analysis, combination, education and outreach. Funding agencies would receive more scientific return, and a positive image, from their initial investment leading to the production and the first analysis of preserved data. An international Study Group reviewed the current state of data preservation in HEP: summarised physics arguments for preservation, outlined possible data preservation models, described technologies and facilities and examined the aspects of supervising and governing data preservation. The preliminary conclusions of the Study Group are: 1. Data preservation beyond the end-date of experiments opens up future scientific opportunities. Given the present status of experimental programs at most facilities, an urgent and vigorous action is needed to ensure data preservation in HEP. 2. Different levels of data preservation and usability are possible. The preservation of the full analysis capability of experiments is recommended, including the preservation of reconstruction and simulation software. A dedicated project in each experiment is needed to assess the corresponding technological requirements. 3. The technological aspects of …